Greptile Summary

This PR is one layer in a broader "unshim stack" that consolidates version-specific Spark shim sources into shared helpers. It removes duplicate shim sources for Spark 3.3.1–3.4.x that are now provided by new helper modules introduced in the preceding stack layer.

• 11 SparkShimServiceProvider boilerplate files deleted (spark331–spark344, including Databricks variants); their version registration is now handled by shared infrastructure from the previous stack layer.
• Several classes migrated off Spark's internal Logging trait (RapidsShuffleIterator, RapidsCachingReader, HiveFileUtil, GpuWindowGroupLimitingIterator) in favour of direct SLF4J, which is required when source files are compiled into multiple shim JARs without access to Spark internals.
• OrcProtoWriterShim refactored to reflection to support both org.apache.orc.protobuf and com.google.protobuf variants across ORC library generations; GpuGroupedPythonRunnerFactory converted from case class to class extends Serializable with explicit argNames parameter; blank lines added to Python runner files for binary-dedupe alignment.

Confidence Score: 4/5

The PR is a structural cleanup with no intended behavioral change; the full-stack build was validated tree-equivalent to the pre-split stack, so merging carries low risk.

The deletions are mechanical and the refactorings (Logging → SLF4J, WriteFilesExecRule delegation, shim provider removal) are straightforward. The OrcProtoWriterShim reflection rewrite is the most complex change: it is logically correct but surfaces an opaque error message when neither protobuf library is resolvable at runtime. The GpuGroupedPythonRunnerFactory API break (removed argNames default) is an internal shim concern that should be covered by the full-stack build.

sql-plugin/src/main/spark340/scala/com/nvidia/spark/rapids/shims/OrcProtoWriterShim.scala deserves a second look for the reflection-based proto dispatch and its error-path diagnostics.

Important Files Changed

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["SparkShimServiceProvider (deleted per version)\nspark331 / 332 / 332db / 333 / 334 / 340 / 341 / 341db / 342 / 343 / 344"]
    B["Shared helper module\n(previous stack layer #15053)"]
    A -->|"replaced by"| B

    C["Spark.internal.Logging (removed)"]
    D["Direct SLF4J logger"]
    C -->|"replaced by"| D
    C --> RapidsShuffleIterator
    C --> RapidsCachingReader
    C --> HiveFileUtil
    C --> GpuWindowGroupLimitingIterator
    RapidsShuffleIterator --> D
    RapidsCachingReader --> D
    HiveFileUtil --> D
    GpuWindowGroupLimitingIterator --> D

    E["OrcProtoWriterShim (static CodedOutputStream)"]
    F["OrcProtoWriterShim (reflection over protoApis)\nSupports org.apache.orc.protobuf + com.google.protobuf"]
    E -->|"refactored to"| F

    G["WriteFilesExecRule (new shared object)\nWriteFilesExecShims.exec → GpuWriteFilesMeta"]
    H["Spark332PlusDBShims.getExecs"]
    I["Spark340PlusNonDBShims.getExecs"]
    G --> H
    G --> I

_{Reviews (1): Last reviewed commit: "Remove old Spark 3.3.1 through 3.4 shim ..." | Re-trigger Greptile}